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ABSTRACT 



Inspection of solder paste on a printed circuit board using a 
histogram of an image before printing (pre -application 
image) to normalize an image after printing of the printed 
circuit board (post-application image) is described. Existing 
lighting and optics used for alignment of the screen-printing 
stencil to the printed circuit board are used for the solder 
paste inspection. 
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METHODS AND APPARATUSES FOR IN- 
LINE SOLDER PASTE INSPECTION 

This application is a continuation in part of application 
Ser. No. 08/769.739 filed Dec. 19, 1996, now U.S. Pat. No. 
5,912,984. 

HELD OF THE INVENTION 

The present invention relates to machine vision and 
automated inspection systems, and more particularly to a 
method and apparatus for inspection over a sequence of 
process steps. 

BACKGROUND OF THE INVENTION 

The process of assembling printed circuit boards (PCBs) 
generally requires solder paste to be applied to the printed 
circuit board before components, such as integrated circuits 
and discrete devices, are placed and ultimately secured onto 
the board. Initially, the sticky solder paste temporarily holds 
the components in place on the PCB until the solder paste is 
melted or reflowed. When the solder paste is reflowed it 
forms both mechanical and electrical connections between 
the components and the board. If there is not enough solder 
paste, components may be inadequately mechanically and/or 
electrically connected to the PCB. If there is too much solder 
paste or if solder paste is deposited in the wrong place, an 
extra or incorrect electrical connection such as a short circuit 
may result, generally referred to as a "solder bridge". 

Automated processes for fabricating PCBs are generally 
known, including process steps for applying solder paste to 
printed circuit boards using screen printing techniques. Prior 
to printing, a thin metal stencil or template is constructed 
with openings or apertures matching appropriate places on a 
particular PCB where paste should be printed. During print- 
ing the stencil is aligned to the PCB. The solder paste is then 
applied to the PCB through the stencil openings, typically 
using a squeegee to spread the paste. Then the stencil is 
lifted off of the PCB which is transported to its next 
manufacturing process step. 

Automatic inspection or vision systems may be used in 
known processes to detect problems in the printing process, 
including problems associated with the stencil and problems 
associated with the application of paste to the PCBs. Stencil 
blockage may be detectable wherein dried solder paste or 
debris causes stencil openings to clog. It is desirable to 
detect stencil blockage by inspecting the stencil before any 
printed circuit boards are actually misprinted with solder 
paste. Stencil smearing may be detectable wherein solder 
paste smears around the stencil openings on a side of the 
stencil adjacent to the PCB. It is also desirable to detect 
stencil smearing by inspecting the stencil before any boards 
are actually misprinted with solder paste. Misalignment of 
the stencil and PCB may be detectable prior to distribution 
of solder paste by inspection of the stencil position relative 
to the PCB before printing thereon. The PCB can be 
inspected to detect incorrect solder paste distribution after 
printing. Automatic inspection may be used to determine if 
there is too little solder paste, too much solder paste, or 
solder paste in the wrong place on the printed circuit board. 

The automated inspection of solder paste on PCBs is 
inherently difficult to accomplish. The solder paste is diffi- 
cult to identify on the PCB because it is variable in terms of 
its appearance. The paste appearance varies over time, e.g. 
wet paste has a different appearance than dry paste. The 
paste has texture and the gray level reflectance of the paste 
is similar to solder pads with tinning and other circuitry on 
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the PCB. Also, the three dimensional shape of the paste 
causes shadowing, making automated inspection difficult. 
The inherent difficulty is exacerbated by variability from one 
PCB to the next. For example, the color of boards of a single 

S board type in one printing run can vary from hght green to 
dark green or blue. In one printing run, the relative posi- 
tioning of portions or all of a PCB of a single board type may 
vary with respect to the stencil or solder mask on the board. 
Additionally, all of the circuitry on a board is subject to 

jQ dimensional variability. Perhaps most significantly, the sol- 
der paste deposited on a board can obscure solder pads 
beneath it making it impossible to tell either what is beneath 
the solder paste or what is the relative alignment of the 
solder paste to what is beneath it. For instance, if the solder 
paste is exactly the same size and shape as the solder pad 
beneath it, it may be impossible to teU whether there is 
perfect coverage or no coverage at aU. 

Known in-line or image automated inspection systems for 
solder paste inspection do not adequately overcome such 

20 difficulties. Some known systems typically only perform 
inspection based on an image of the PCB captured after the 
solder paste has been deposited thereon (referred to herein- 
after as "post-only" systems). Known post-only systems 
typically use a vision/inspection tool that assigns gray scale 

25 values to pixels of a captured image of a PCB under 
inspection after printing. Pixels are classified as being not- 
paste or paste based on their gray scale value. Pixels are 
aggregated for particular areas of interest on the PCB, in a 
connectivity analysis in which pixels connected to one 

30 another and identified as being paste are labeled as belong- 
ing to the same lump of paste. Characteristics of interest 
related to the aggregation of pixels, such as area, center of 
mass, and size of enclosing rectangle, are determined and 
ultimately used by an operator to adjust parameters of the 

35 printing process to maintain the post -printing product within 
acceptable tolerances. 

Unfortunately, color variation in the underlying areas of 
the PCB negatively affects the inspection process because 
gray scale values will vary according to the color variation 

40 of the underlying portions of the PCB, especially near the 
paste edge where the solder paste is thin. Variability in 
color/composition of the solder paste may also cause gray 
scale variations that negatively affect the inspection process. 
Similarly, known post-only systems may be negatively 

45 affected by lighting, which must be rigorously controlled in 
order to avoid negative affects on the inspection process due 
to shading or reflection of light from components of the 
system. In general, the yield and accuracy of known post- 
only solder inspection systems are not sufficient to make 

50 such systems useful or practicable. 

Many known solder paste inspection systems are stand- 
alone (i.e. not "in-line") stations dedicated to performing 
solder paste inspection apart from and subsequent to the 
screen printing process step(s). Such stand-alone systems 

55 include optics and lighting that are optimally configured for 
the inspection of solder paste. The optics and lighting 
dedicated to solder paste inspection represent costly, redun- 
dant hardware in the PCB fabrication process flow in that 
similar optics and Ughling are necessary upstream in the 

60 process flow to effect registration or aflgnment in the screen 
printing stage. Furthermore, stand-alone solder paste inspec- 
tion systems typically incorporate laser stripers and other 
very expensive and sophisticated hardware that is often 
difficult to justify in a cost/benefit analysis. Stand-alone 

65 systems disadvantageously introduce additional machine 
set-up time and maintenance, require additional factory floor 
space to implement, and introduce additional undesirable 
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handling of PCBs to convey them through the stand-alone 
inspection stage. Generally, a stand-alone solder inspection 
system will represent a significant overall reduction in 
throughput in a PCB fabrication line. 

SUMMARY OF THE INVENTION 

The present invention provides methods and apparatuses 
for identifying a first region within a second-slate image 
using at least one first-state characteristic. In a preferred 
embodiment after the first region has been identified, the 
invention determines how the first region has changed 
within the second-state image relative to the first region in 
a first-state image. 

The first-state image and the second-state image differ 
from one another, where the first -stale image and the second- 
state image are, for example, images of a product at different 
stages in a manufacturing or printing process. 

The first region may or may not differ between the 
first-state image and the second-state image, but the differ- 
ence or lack thereof is detectable by the methods and 
apparatuses of the invention. The first region is, for example, 
an uncovered pad region of a printed circuit board in the 
first-state image and a partially covered solder-paste pad 
region, a completely covered solder-paste pad region, or an 
uncovered pad region in the second-state image. 

The first- state characteristic represents an aspect of the 
first region within the first-state image, such as the area of 
the pad region. In a preferred embodiment, the first-state 
characteristic is a first-state histogram, which represents 
more than one aspect of the first region. However, the 
first-state characteristic can be any fonm of extracted data. 

The invention detects the first region by generating a 
second-state histogram of the second-state image and then 
by using the first-state characteristic to augment the repre- 
sentation of the first region within the second-state histo- 
gram relative to at least one other region in the second-state 
histogram. Once the first region is enhanced within the 
second -state histogram, the first region is more easily iden- 
tifiable and can be used to ascertain further information 
about the second-state image. Details of the second-state 
image and regions within the second-state image can be 
detected, including the extent of the first region as well as the 
existence and the extent of other regions. 

In one embodiment, the invention enhances the first 
region by selectively combining the first-state histogram 
with the second-state histogram to produce an augmented 
second-state histogram. In the augmented histogram, an 
intensity value of each bin within the first region is the 
greater of the intensity values of the first region from either 
the first-state histogram or the second-state histogram. 

In another embodiment, the invention enhances the first 
region relative to one other region by zeroing the intensity 
values of at least one other region within the second-state 
histogram to produce an augmented second -slate histogram. 
To zero the intensity values of at least one other region, the 
invention calculates the area of the first region of in the 
first-state image. Then, the invention identifies a region in 
the second-state histogram that substantially contains the 
area of the first region of the first-state image. For at least 
part of the pixels in the area substantially outside the 
identified region, the invention zeros the intensity values of 
those pixels. The non-zero intensity values that remain in the 
augmented second-state histogram include the pixels that 
represent the first region. In one embodiment, the invention 
zeros all of pixels from areas outside of the area of the first 
region. Thus, the non-zero intensity values that remain have 
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an area that substantially equals the area of the first region. 
Further in a preferred embodiment, the area is determined 
from the first-slate histogram. 
Also in a preferred embodiment, the secx)nd-state histo- 
5 gram is enhanced by both selectively combining the first- 
state histogram with the second-state histogram and remov- 
ing a portion of the intensity values from the second-state 
histogram. 

It is recognized by the invention that information about 
any region, such as the extent of a paste region or presence 
of a region, for example, within the second-slate image can 
be determined using a first-state characteristic, such as a 
first-state histogram, of the first-state image. 

It is recognized by the invention that information which is 
typically difficult to discern from the second -state image can 
more easily be determined using the information derived 
from the first-state characteristic. Consequently, the inven- 
tion is particularly useful for identifying otherwise poorly 

^ defined regions within the second-state histogram. 

The present invention can be advantageously used with a 
screen printing process, the invention identifying solder 
paste and determining if the paste covers a pad region of a 
PCB. In this application, a pre-paste application histogram 
(first-state histogram) of a pre-paste application image (fiist- 
state image) augments a post-application histogram (second- 
state histogram) of a post-paste application image (second- 
state image) so that a threshold of the paste/pad region 
transition is easily identified within the augmented second- 
state histogram. Optionally, the post-paste application image 
is binarized using the paste/pad threshold, thereby producing 
a binary image that segments the uncovered pad region from 
the rest of the image. 

In other aspects, the method is enhanced by using a model 

35 of the first-state image to analyze the second-state image or 
the optionally generated binary image, e.g. a model of a 
position and shape of the pad area. The model of the 
first-stale image is aligned with the second-stale image or 
the optionally generated binary image and then compared. 
The comparison yields whether the aligned region(s) are 
different, and therefore, whether the region(s) have changed 
between the first-state image and the second-state image. 
For example, the comparison yields the extent of the paste 
on the pad region. 

45 In a preferred embodiment, an optional model is regis- 
tered with the first-state image to isolate selected region(s) 
of the first-state image. The model can, for example, window 
selected regions, such as pad regions. The model aligned 
with the first-state image produces an aligned first-state 

50 image upon which the first-state characteristic, such as a 
histogram, is generated. The first-state characteristic sub- 
stantially represents only the regions isolated by the model. 

This enhancement of the first-state characteristic, (i.e. the 
selection of regions represented within the first-state 

55 characteristic) makes the calculation of the area of the 
selected region, such as the pad region, from the first-state 
image more simple and reliable. Consequently, the area 
examined in the second-state histogram will also more 
reliably represent the pad region. 

60 Features of the invention include provision of an auto- 
mated inspection system, optionally used in-line during 
screen printing of PCBs, that avoids redundant hardware and 
associated cost by effecting inspection using existing fight- 
ing and/or optics present in the screen printing equipment 

65 for alignment purposes. The technique implemented in sev- 
eral embodiments of the invention normalizes each board 
with a histogram of that same board without paste and 
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therefore avoids problems associated with prior art post-only 
inspection systems resulting from variations in board color, 
solder mask shifting and/or board or component dimension 
variability. Through the use of the histogram of the pre- 
applicalion image as opposed to the storage of the pre- 5 
application image, each inspection requires less memory 
resources, and therefore allows an increased number of 
inspections per board. 

Various problems with prior art solder paste inspection 
systems are overcome by the method and apparatuses 30 
according to the invention, including problems resulting 
from pads being covered by paste so that they are not visible 
or otherwise distinguishable, and problems resulting from an 
inability to distinguish among pads that have no paste on 
them and pads that are exactly 100% covered by paste. 15 
Several embodiments of the methods and apparatuses 
according to the invention benefit from implementation of a 
data gathering method that achieves the benefits of the 
subtraction procedure without the constraints of the regis- 
tration mechanism as taught by other embodiments of the 20 
invention. 

BRIEF DESCRIPTION OF THE DRAWING 

The foregoing and other features and advantages of the 
present invention will be more fully understood from the 25 
following detailed description of illustrative embodiments, 
taken in conjunction with the accompanying drawing in 
which: 

FIG. 1 is a schematic diagram of components of a screen 
printing system as known in the art; 

FIG. 2a is an illustration of stencil apertures correspond- 
ing to printing regions for receiving solder paste; 

FIG. 2b is an illustration of blocking regions where solder 
paste blockage is looked for in the stencil apertures of FIG. ^5 
2a; 

FIG. 2c is an illustration of smearing regions where 
smearing is looked for in the stencil apertures of FIG. 2a; 

FIG. 2d is an illustration of runtime images of the 
apertures of FIG. 2a including the blocking regions of FIG. 40 
2b and the smearing regions of FIG. 2c; 

FIG. 3 is a flow diagram of process steps implementing 
stencil side inspection for a method and apparatus for in-line 
solder paste inspection according to the invention; 

FIG. 4 is a flow diagram of process steps implementing 
printed circuit board side inspection for a method and 
apparatus for in-line solder paste inspection according to the 
invention; 

FIG. 5a is an illustration of a pre-image with defined 
on-pad regions and off-pad regions; 

FIG. 5b is an illustration of an on-pad difference image 
representative of the position of solder paste on solder pads; 

FIG. 5c is an iUustration of an off-pad difference image 
representative of the position of solder paste on areas of the 
board other than the pads; 

FIG. Sd is an illustration of an image representative of 
solder paste on the printed circuit board; 

FIG. 6 is a flow diagram of process steps implementing 
printed circuit board side inspection for an embodiment of 
the method and apparatus for in-line solder paste inspection 
according to the invention; and 

FIG. 7A is an illustration of a pre-paste application 
histogram of a windowed pad region in a prc-application 
image of a PCB; and 65 

FIG. 7B is an illustration of a post-application histogram 
of a PCB after application of paste; and 
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FIG. 7C is an iUustration of an augmented histogram 
generated by removing the intensity values of the off-pad 
regions from the post- application histogram of FIG. 7B; and 

no. 7D is a representation of an augmented histogram 
generated by selectively combining the pre-paste application 
histogram of FIG. 7A and histogram of FIG. 7C. 

DETAILED DESCRIPTION 

The present invention is useful in the context of many 
multi-step printing processes and manufacturing processes, 
including solder paste screen printing systems known in the 
art. Solder paste printing using screen printers is typically 
effected on bare printed circuit boards (PCBs) in a manu- 
facturing production line prior to placement of components 
on the board. The bare PCB is placed on or transported to a 
screen printing station 10, such as illustrated in FIG. 1, 
whereat the PCB 12 is held firmly in place on a support 14 
by either vacuum or mechanical means. The screen printing 
station includes an upper portion that houses a stencil 16 
having apertures that correspond to areas of interest on the 
PCB, such as solder pads, where solder is to be disposed. 

Optics 18, such as movable lelecentric optics, a CCD 
camera 20 and flash lighting 22 aU known in the art are used 
in conjunction with a vision system, such as a Cognex 5000 
Series vision system and vision tools known in the art, to 
capture images of stencil 16 and board 12 fiducials to effect 
registration, i.e. alignment of the stencil to the board. It 
should be noted that the placement of lighting 22, the camera 
20 and optics 18 is a function of the particular vision system 
configuration and application as known in the art, and as 
provided herewith FIG. 1 is not intended to be illustrative of 
any particular implementation or application. The illustra- 
tive system effecting the functionality described hereinafter 
incorporates a vision system processor 23 receiving signals, 
typically captured image signals, from the camera 22. The 
vision system processor 23 has access to a memory 25 in 
which it stores information processed in effecting the inspec- 
tion system according to the invention. 

Generally, in the screen printing process registration or 
alignment between the stencil and the PCB is accomplished, 
such as by a method involving registration of fiducials as 
described in commonly owned U.S. Pat. No. 5,550,906 
which is incorporated herein by reference, and the stencil 16 
is lowered or actuated to a position proximate to the PCB, 
The solder paste is deposited on a side of the stencil 16 away 
from the PCB and is pushed through apertures or openings 
in the stencil onto solder pad sites on the PCB 12 by means 
of a squeegee (not shown), that traverses the stencil at an 
appropriate speed while exerting appropriate pressure. The 
stencil 16 is then lifted or actuated away from the PCB 12, 
leaving solder paste deposits on pad sites corresponding to 
the apertures in the slendl. 

More specifically, in an illustrative in-line solder paste 
inspection implementation according to the invention, prior 
to closing the fixture (i.e. moving the stencil proximate to the 
PCB for printing), the vision system captures images of 
areas of interest on both the stencil and board (the side of the 
stencil facing the board being somewhat of a "mirror image" 
of the board side facing the stencil). Substantially 
simultaneously, the vision system captures images of fidu- 
cials on the stencil and board for alignment purposes, and 
images of sites that are subject to inspection on the stencil 
and the board. Further, after printing of the solder paste on 
the PCB the stencil is aauated away from the PCB and 
post-printing images of the stencil and the PCB are captured 
to effect the inspection process as described in detail here- 
inafter. 
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Inspecting a given stencil and a given board may involve For each uniquely identified aperture a blockage region is 

multiple images depending on the image size, the stencil and specified during training, as illustrated in FIG. 2b, The 

board size and the number of sites on a stencil and board that blockage region 30, 30' is a region interior to the aperture 28, 

the user wants to inspect In typical usage, multiple sites on 28' which will be inspected for occlusion resulting from 

each of the stencil and the board are inspected. Therefore, 5 residual solder paste remaining on the stencil. Additionally, 

capturing of an image on each of the stencil and the PCB, for each uniquely identified aperture a smear region is 

both before and after printing, is effected as many times as specified during training, as illustrated in FIG. 2c, The smear 

is necessary to capture images of each of the relevant sites region 32, 32' is a region exterior to the aperture 28, 28' 

on a particular board in a particular inspection cycle. In a which will be inspected for the presence of solder paste 

typical PCB manufacturing process it is most expedient to resulting from residual solder paste remaining on the stencil, 

"tile" the stencil and PCB into a plurality of sites so that only During a run-time inspection of the stencil as described in 

images of relevant portions of the stencil and PCB are detail hereinafter, smearing and blockage might appear as 

captured during stencil and board inspection in a given illustrated in FIG. 2i/. The extent of the open aperture 34, 34' 

cycle. Tiling minimizes memory storage requirements for a impinged upon or restricted by blockage 36, 36' 

given substrate or board under inspection. ^^^^^ detected as it resides within the specified 

Furthermore, it is preferable that the tiles of the stencil blockage region. Likewise, smearing 38, 38' of solder paste 

and PCB be captured in a round-robm fashion so that not ^jj ^^^^^^^ ^^^^^^ ^^^^^^ ^^^^^^^ the 

every site is inspected on a given board in a given cycle^ ^^^^^ ^^.^^ ^^^-^ -^^^ ^^^^^ .^^ 

That IS, processing and fabncation hne spe«^ Although not iUustrated in FIGS. 2a-2d, the blockage region 

when inspection according to the invention is implemented , - , ^j ^ T 

such that only a portion of aU of the relevant sites are 20 smear region may be specified so as to incorporate a 

captured and inspected on a given board, while another ^on't care region of several pixels between the outer edge 

portion of the relevant sites are inspected on a successive t^e blockage region and the inner edge of the smear 

board and stiU another portion arc inspected on still another region. Such a don't care zone is useful in that it removes 

board. In this manner, an inspection cycle (i.e. inspection of from consideration the visual appearance at the stencil edge, 

every site on a given board type) is effected over a plurality 25 ^^^^^ appearance at the stencil edge may be highly 

of boards under inspection. The round robin inspection in variable and extremely sensitive to lighting changes, squce- 

conjunction with tiling facilitates better statistical analyses. gee position during printing, and aspects of the mechanical 

In the illustrative embodiment of the invention described process used to form the aperture in the stencil, 

hereinafter, stencil inspection is described first, however, it The stencil inspection process flow in the solder paste 

should be appreciated that PCB inspection as described 30 inspection procedure according to the invention is illustrated 

hereinafter can be effected prior to stencil inspection. In the in FIG. 3. Initially an image of the stencil underside or side 

present implementation it is preferable to consider the to be positioned adjacent to the PCB is acquired 40 before 

stencil first because certain efficiencies can be achieved in solder paste is printed, to obtain pre-printing image data (at 

the training and inspection processes. For example, infor- times referred to herein as "stencil pre-image data"). The 

mation obtained in a training process for inspection of the 35 pre-image can be acquired immediately before printing a 

stencil can optionally be used in a training process for board in-process or it can be obtained well in advance of the 

inspection of the board, as described in detail hereinafter. screen printing process. While the pre-printing image or 

Similarly, information derived in a registration process for pre-image data in this illustrative embodiment is acquired 

the stencil can be used as a starting point to simplffy and in-line using the camera and optics available for registration 

enhance the registration process for the PCB in the PCB 40 of the stencil and PCB, as discussed hereinbefore, the data 

inspection process, also described in detail hereinafter. can be acquired as data files from automated machinery used 

Prior to undertaking any inspection of the stencil, the to fabricate the stencil (or PCB). Similarly, the pre-image 

vision system must be trained, as known in the art. Training data can alternatively be synthesized from a geometric 

is handled "off-line'', i.e. prior to any inspection run, and in description of the stencil, including information about the 

the case of stencil inspection basically involves constructing 45 location and size of each aperture, combined with calibration 

a model of the stencil by identifying stencil characteristics information required to calibrate the geometric description 

based on a "golden sample'' or known good stencil. The of the stencil to pixel size(s) and aspect ratio(s) of the 

stencil characteristics identified during training are related to camera imaging system that is used to acquire post-printing 

sites of interest, in this case apertures in the stencil corre- image data. 

spending to the locations of solder pads on the PCB. 50 Inspecting a stencil may involve multiple images depend- 

Training in this illustrative embodiment is effected using ing on the image size and stencil size and the number of sites 

vision tools with training capabilities, such as Cognex Blob on a stencil that the user wishes to inspect, thus acquisition 

Analysis Tool, which runs on the vision system used in the or construction of stencil pre-image data is effected for each 

inspection system. site on a stencil that is to be inspected. Subsequent steps in 

For each aperture 28, 28', such as illustrated in FIG. 2a, ss the inspection process are performed as many times as 

a data structure is constructed during training including a necessary to effect inspection of each site for which pre- 

unique identifier for the aperture and characteristics of the image data (and post-image data) is acquired, 

aperture. Characteristics relating to and stored in association The image acquired by the camera imaging system is 

with each uniquely identified aperture include an aperture registered 42 to the model of the stencil stored during 

width, aperture height, and an X-Y location of the center of 60 training. This initial registration effectively aligns the 

the aperture. For apertures with comer rounding (e.g. 28'), selected sites where the apertures are on the stencil image 

radial values indicative of curvature are also stored. When acquired before printing (the prc-imagc), with the selected 

the Cognex Blob Analysis Tool is used for training, center of sites of the apertures on the model stencU image stored 

mass as determined by the Blob Analysis Tool as known in during training. Registration compensates for any differ- 

the art, is used as the X-Y location. Those skilled in the art 65 ences in the coordinate systems of the respective acquired 

will appreciate that median, mode or center of boundary box images. This initial registration is accomplished by perform- 

can also be used as the X-Y location. ing two one-dimensional registrations, one in the X and one 
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in the Y, on projections, as known in ihe art, of the naodel and value (or range of gray values) as representative of paste and 

stencil images. Registration is achieved when there is maxi- another gray value (or range) as representative of other than 

mum correlation between the projections in each dimension. paste, i.e. stencil background. The alignment effected by 

The initial registration can alternatively be effected by vision registration between the prc-image and the post-printing 
tools known in the art, such as described in U.S. Pat, No. S binary image clearly indicating paste locations, facilitates 

5^48,326 entitled EFHCIENT IMAGE REGISTRATION detection of paste in the blockage and smear regions 

which is incorporated herein by reference. Registration can (defined during training). 

be implemented as well, with other vision tools using The difference image (if necessary) or the binary image 

fiducials on the stencil. contains informatio5_clearly-iridicnivc-of4hc4o^^ 

After the pre-image data is acquu-ed and after registration ^^soldefpiSte on the stencil or in the aperture(s). It is desirable 

of the pre-image and the model image, printing of the solder to separately process aperture information in order to detect 

paste can be effected. blockage and stencil background information in order to 

Thereafter, the camera acquires an image of the stencil delect smearrA^ iriagr^i i£;^^med^ 

underside after solder paste 44 has been printed on the mfonnadon^to„define„inrap; ^rc a ^^ 

circuit board (the "stencil post-image"). The stencil post- should be appreciatedt^:thc_niasl£:can.aIso.be.geDcratc?by 

image corresponds to the same stencil or section of stencil ^thresholding the p rerimage. The mask is a pplied SO to:tBP 

that was acquired before the screen printing of the solder /difference (or binary) image to ig^^ut the aper ture-ar&?> 

paste . from the.stendl background area so lhat those^two groupsof T 

The stencil pre-image and the stendl post-image are then 
registered to one another 46. RegistraUon of the prc-image Analysis of the in-aperture region and the off-aperture 
to the post-image can have several degrees of freedom, such region 52 can then be effected by using a connected com- 
as translation (2 degree), rotation, scale, skew, etc. In the pouents analysis as known in the art. Alternatively, Cogn- 
present implementation, rotation, scale and ^ew are mini- ®x's Blob Analysis Tool can be used to measure paste 
mizcd and registration (both prc-image to model image and distribution on the stendl. Cognex's Blob Analysis Tool 
pre-image to post-image) is effected for translation only. A provides information such as area, center of mass, 
preferred method for pre-image to post-image registration is orientation, moments, and extent. In this illustrative 
to use the technique of Efficient Image Registration (U.S. embodiment, area measurements of blockage and smear are 
Pat. No. 5,548,326) which can align two images of an object measurements of primary interest. The regions of interest 
that has undergone a manufacturing process. However, other applying the Blob Analysis Tool arc the registered stencil 
techniques such as Cognex's Normalized Correlation Search ^° openings for paste blockage and around the stencil openings 
tool, Cognex's Caliper Tool, or Cognex's CNLPAS tool for paste smearing. The exact locations of the openings are 
could also be used to align the two images. It should be known because the aperture locations are known from the 
appreciated that pre-image to post-image registration works pre-image acquired before screen printing and the relative 
best if there are no changes in lighting between acquisition position changes of the apertures are known from the 
of the respective images. '^^ post-image acquired after solder paste screen printing. 

The registration step 46 allows information from the bare Printed circuit board inspection described in detail here- 

stencil (pre-image) to be combined with information from in^^er is effected in a substantially similar manner as the 

the stencil after screen printing with paste (post-image). The stencil inspection described in detail hereinbefore, 
two aUgned images, i.e. the pre-image and the post-image, As with the stencil, prior to undertaking any inspection of 

can be effectively combined by optionally subtracting one the PCB, the vision system must be trained, as known in the 

from another 48 yielding a difference image with pixel art. Training for the PCB is also handled "off-line'' prior to 

values that arc signed quantities (i.e. positive or negative). a^ny inspection run. Like in the case of stencil, PCB training 

The positive value of pixels in the difference image represent basically involves constructing a model of the PCB by 
blockage when those pixels appear in the blockage region of 45 identifying PCB areas of interest and characteristics. The 

the aperture. The negative value pixels represent smear PCB characteristics identified dm^ing training are related to 

when those pixels appear in the smear region. sites of interest, which in the PCB case arc solder pads 

The difference image is then separated into two images for corresponding to apertures in the stencil, 
separate processing of positive value pixels and negative As with the stencil side, PCB inspection training is 
value pixels. An expeditious method of separating the dif- 50 effected by acquiring an image of a known good entity, in 

ference image is to duplicate the image to form a positive Ihis case a PCB, and constructing a model of characteristics 

value image and a negative value image, and then replace all of interest, in this case solder blobs on solder pads, 

of the negative pixels in the positive value image with dont Generally, PCB training is used to define a model image 

cares (i.e. zeros) and replace all of the positive value pixels based on a registration window within a field of view. The 
in the negative value image with don't cares. All pixels in the 55 model of the PCB constructed during training will result 

negative image can then be replaced with their absolute from consideration of where the solder is supposed to go as 

values for mathematical convenience, and the positive value defined by a baseline location of the pads of interest and 

image and the absolute value image can be separately Iocation(s) on the pad where solder is properly deposited, 
processed. As with the stencil side trainmg, training for the PCB in 

The subtraction step 48 is optional because it may not be 60 this illustrative embodiment is effected using vision tools 

necessary to subtract and obtain a difference image with training capabilities, such as Cognex Blob Analysis 

(representative of difference in gray value) in order to clearly Tool, which runs on the vision system used in the inspection 

distinguish locations where there is paste from locations system. 

where there is no paste. In situations where there is signifi- For each registration window selected within a field of 
cant contrast in color (and consequently significant disparity 65 view, a data structure is constructed during training includ- 

in gray value) between paste and non-paste areas, a binary ing a unique identifier for the pads of interest. Further, 

image can be generated based on discriminating one gray characteristics for each of the pads are stored in the data 
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Structure in association with the pads unique identifier, linear changes in brightness to be ignored when registering 

including: the pad's width, height, X-Y location, and any the model to the pre-image. As with the initial registration in 

comer curvature information. As with the stencil training the stencil inspection, initial registration in the PCB inspec- 

procedure, PCB training for pad information is effected tion can be implemented as well, with other vision tools 

using a vision tool such as Cogncx's Blob Analysis tool or s using fiducials on the stencil. Similarly, the initial registra- 

other vision tool having u-aining capabUities as known in the ^an be effected by vision tools known in the art. sudi as 

art. described in the referenced U.S. Pat. No. 5,548,326 entiUed 

In addition to information relating to the pads, training for EFFICIENT IMAGE REGISTRATION 

the printed circuit board inspecUon may involve gathering prc-imagc data is acquired 60 and registered 62, 

mformation relatmg to solder bncks or blobs deposited on a „ ^^^^^ ^^^^ ^ ^^^^^^ 

known good PCB. In such a case, where certain of the " 

information relating to the solder is a function of daaracter- P^^^ ^^en printed on the circuit board, 

istics of the stencil used to deposit the solder, the efficiency camera acquires an image of the PCB (the "PCB 

of the process can be enhanced by using, in the training for post-image") 64. The PCB post-unage corresponds to the 

the PCB, certain information obtained in training for the ^ame PCB or section of PCB that was acquired before the 

stencU. Specifically, a given solder brick can be identified ^^^^^^ P"°^*°S colder paste, 

and correlated to a known aperture on the stencil already The PCB pre-image and the PCB post-image are then 

trained for. Width and height information for respective registered to one another 66. Again, a preferred method for 

apertures can be used as width and height information for pre-image to post-image registration is to use the technique 

corresponding solder deposiUons in training for the PCB. Efficient Image Registration (U.S. Pat. No. 5,545,326) 

Similarly, the X-Y location of the respective apertures can be which can align two images of an object that has undergone 

used for a starting point for determining the X-Y location of a manufacturing process. However, other techniques such as 

the corresponding pads in gathering characteristics or train- Cognex's Normalized Correlation Search tool, Cogncx's 

ing for the PCB. The training process for the PCB may be CaUper Tool, or Cognex's CNLPAS tool could also be used 

implemented to utilize virmally all of the characteristics images. The rcgisU-ation step 66 allows 

relating to and stored in association with each uniquely information from the PCB before printing (PCB pre-image) 

identified aperture, including the aperture width, aperture lo be combined with information from the PCB after screen 

height, and X-Y location, and for apertures with comer printing with paste (PCB post-image), 

rounding the radial values indicative of curvature can be The two registered or aligned images, i.e. the PCB pre- 

used. It should be appreciated that the training information 3Q image and the PCB post-image can be effectively combined 

relating to the stencil is a "mirror image" of the information by optionally subtracting one from another 68 yielding a 

as it would relate to the PCB, therefore in order to use the difference image with pixel values that are signed quantities 

stencil information it must be processed accordingly. It (i.e. positive or negative). Since solder paste is generally 

should also be appreciated, as previously mentioned, that darker than printed circuit board pads and brighter than 

training for the PCB can be done as with the stencil, by using 35 printed circuit board background, the positive value pixels in 

a vision tool such as Cognex's Blob Analysis tool or other the difference image represent solder paste on the pads and 

vision tool having training capabiUties as known in the art. the negative value pixels represent solder paste off the pads 

The PCB inspection process flow is illustrated in FIG. 4. and on the printed circuit board background. Zero value 

Initially an image of the PCB is acquired 60 before solder pixels may result in the difference image, which represent 

paste is printed, to obtain pre-printing image data (at times 40 either locations without solder paste or locations where the 

referred to herein as "PCB pre-image data"). As with the solder paste happens to have the same gray value as the 

stencil, the image can be acquired immediately before printed circuit board proximate to it. Zero value pixels are 

printing a board in-process or it can be obtained well in "don't care" pixels, i.e. of no interest, 

advance of the screen printing process. While the pre- The difference image is then separated into two images for 

printing image or PCB pre-image data in this illustrative 45 separate processing of positive value pixels and negative 

embodiment is acquired in-Une using the camera and optics value pixels. An expeditious method of separating the dif- 

available for registration of the stencil and PCB, as dis- ference image is to duplicate the image in order to form a 

cussed hereinbefore, the data can be acquired as data files positive value image and a negative value image. This is 

from automated machinery used to fabricate the PCB, as accomplished by replacing aU of the negative pixels in the 

known in the art. 50 positive value image with don't cares (i.e. zeros) and replac- 

Inspecting the PCB, as with the stencil, may involve ing all of the positive value pixels in the negative value 

multiple images depending on the image size and board size image with dont cares. All pixels in the negative image can 

and the number of sites on the board that the user wishes to then be replaced with their absolute values to form an 

inspect. Thus acquisition or construction of PCB pre-image absolute value image. Thus the positive value image con- 

data is effected for each site on the board that is to be 55 stitutes a solder paste/pad image representative of solder 

inspected. Subsequent steps in the inspection process are paste on pads and the absolute value image constitutes a 

performed as many times as necessary to effect inspection of non-pad/paste image representative of solder paste off the 

each site for which pre-image data (and post-image data) is pads and on the printed circuit board background, 

acquired. As with the stencil side processing, the subtraction step 68 

The pre-image, acquired by a camera imaging system, is 60 is optional bccaiise it may not be necessary to subU^act and 

registered 62 U> the model of the PCB stored during training. obtain a difference image (representative of difference in 

This initial registration effectively aligns the selected sites of gray value) in order to clearly distinguish locations where 

interest on the PCB image acquired before printing (the there is paste from locations where there is no paste. In 

pre-image), with the selected sites of interest on the model situations where there is significant contrast in color (and 

PCB image stored during training. This initial registration 65 consequently significant disparity in gray value) between 

can be efficiently undertaken using a vision tool effecting paste on pad and non-pad/paste areas, a binary image can be 

normalized correlation as known in the art, which allows generated based on discriminating one gray value (or range 
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of gray values) as representative of paste on pad and another image, illustrated in FIG. Sd, represeatative of solder paste 

gray value (or range) as representative of other-than-pad 88 on the printed circuit board. 

paste areas, Le. PCB background. The alignment effected by The single binary image is then analyzed 74 using a 

registration between the pre -image and the post-printing connected components analysis method as known in the art, 
binary image clearly indicating paste locations, facilitates S or by using a known vision tool to measure paste distribution 

inspection and detection of paste locations (as defined and/or to determine information regarding the solder paste, 

during training). such as area, center of mass, orientation, etc. From the paste 

The difference images (if necessary) or the binary image area distribution and two sets of image registrations, other 

contains informatioiij^arlylindicaiiYe^o^ H useful quantities can be derived, as known in the art, such as 
^solder paste. It isfd esira blc to inask outThe pads from the fio indications of bridging between solder blobs and print 

.PCB background so that it is possible-to separately process offsct(s). 

jpad information and non-pad pr background Moimatiq Bridging 90, the misplacement of solder in a manner that 

That is, for purposes of further processing and analysis to results in an erroneous electrical connection between adja- 

(2]deternninfe^to~Jpca^tw and;th e-Iocati on:of cent pads or between a pad and conductive surface proxi- 
^solder off of~the_padS|(Iikely causing bridges between fine 15 j^^te thereto as illustrated in FIG. Sdy is determined by 

pitch or Closely spaced pads), masksjireused^to^ calculating the boundary of the paste region and measuring 

[and independently prbc^ pad inform its proximity to neighboring pads. 

Armasjris'genefa ahd a pplied -70 p^nt offset 92, also illustrated in FIG. Sd, is a deviation 

u^ng PCB training information to define on-pad re^^ ju position between where the paste was actually printed 

and off-pM regions 82 ^ Masking in ^gjative to where it was supposed to be printed as defined at 

this illustrat ive embodiment is based on the relatiy e gray trainuig time. There are two sets of information generated 

\valueJfareasjo^^ and is accomplished^y_se~tting a relating to two "types^' of print offset in the system according 

thresholdon the PCB pre-image and d^ignating all pixeMn (q the invention. 

^^i^Sxg z g^y^ ^ ^Q^^ calculation relates offeet of the center 

thre^Maio^ieon-padpgi^^^ designatmga^l pixels 25 ^^^^ individual solder blobs to the center 

with gray values less than (or equal to) the threshold to be respective or corresponding solder pad, 

off-pad regions 82. Optionally, a connected components determining ofiket therebetween. The first print offset cai- 

analysis as known in the art can be effected on the on-pad ^^^^^^^^ available an offset for each of the individual 

regions to ensure that the pads are a mmimum size. solder blobs 

In a combining operation 72, the on-pad mask is logically ^° A second print offset calculation determines an aggregate 

ANDed with the solder paste/pad image to form an on-pad ofiket of all of the solder paste blobs inspected relative to 

difference image, illustrated in FIG. 5^ representative of the ^j^^j-g ^^^y supposed to be printed on their respective 

position of solder paste on the pads 84. The off-pad mask is colder pads. The second print ofiket calculation effectively 

logicaUy ANDed with the non-pad/pastc image to form an correlates a synthesized, mirrored stencil image to the single 

off-pad difference image, illustrated in FIG. 5c, represenla- binary image representing solder on the board, enabling a 

tive of the position of solder paste on areas of the board other "global" view of the results of the printing process. Such an 

than the pads 86, aggregate view can be achieved as well by taking an average 

The on-pad difference image and the off-pad difference of all of the individual offsets determined according to the 

image are converted, using thresholding operations witb first print offset calculation(s), 

different respective thresholds, to respective first and second Again, Cognex's Blob Analysis Tool can be used to 

binary images. The thresholds are locally adjusted, that is, measure paste distribution on the PCB. Cognex's Blob 

every pixel in the image has its own threshold, based on the Analysis Tool provides information such as area, center of 

pre-image, so that when the pre-image gray scale value is mass, orientation, moments, and extent. Illustrative regions 

nearer to the paste gray scale value the threshold is (linearly) of interest for applying the Blob Analysis Tool on the PCB 

smaller. The paste gray value is determined by statistically are the registered solder pads and center of mass of the 

sampHng known paste pixels at runtime. solder blobs for detecting print offset, and around the solder 

In the present illustrative embodiment, the^ff-pad thresh- pads, i.e. between adjacent solder pads for detecting bridg- 

olds are calculated as a fraction of the filtered post- image ing. The exact locations of the pads are known from the 
and the pn-pad thresholds afe calculated as a fraction of the 50 pre-image acquired before screen printing and the relative 

filtered pre-image, i.e. the gray scale values of each pixel in position of the solder paste to the pads is known from the 

the respective image are each multipHed by a fractional post-image acquired after solder paste screen printing, 

value. The fractional values are determined empirically such It should be appreciated that vision tools implemented in 

that the locally calculated thresholds will be effectively conjunction with the inspection methodology described 
empirically derived as a function of the lighting; ; type of^ 55 hereinbefore can be used to inspect for various other aspects 

£paste, board color and factors affecting the vision system. related to the screen printed material, i.e. other than bridging 

The thresholds can be calculated locally based on a fraction and print offset, such as dross presence (i.e. extra paste not 

of the pre-image or a filtered pre-image gray scale value, associated with any solder pad deposition), total paste cov- 

Local thresholding is described in detail in commonly erage or the like. 

owned, copending U.S. patent application Ser. No. 08/716, jhosc skilled in the art will appreciate that some, or all, 

779 (Atty Docket No. C96-015) filed Sep. 24, 1996 by of the steps of subtraction, masking, thresholding, ANDing 

Nichani entitled SYSTEM OR METHOD FOR DETECT- and ORing described hereinbefore can be combined and 

ING DEFECTS WITH A SEMI-OPAQUE ENCLOSURE, effected as hardware implementations, software implemen- 

which is incorporated herein by reference. Alternatively the lations or a combination thereof, and that although a vision 
thresholds can be an empirically derived fixed quantity. 55 system is described implemented with a Cognex vision 

The respective first and second binary images are in turn system and vision tools, the invention can be implemented 

combined by a logical OR operation to form a single binary on other platforms, such as general purpose computers 
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running software and/or configured with hardware to effect fied to be used to inspect solder paste or adhesives applied 

the functionality described herein. in other ways. That is, eflScient, robust inspection can be 

The local thresholding described herein can also be effected as described hereinbefore using a before image and 

accomplished in various ways, such as by taking the square ^ iftcr image, regardless of how the material added to the 
root of the pre-image, or by computing some linear or S after image is applied 

non-linear fimction of a respective image (i.e. model image, While the method and apparatus described herein use a 

pre-image or post-image), or by using a constant. masking operation that involves application of a threshold to 

It should be further appreciated with respect to the a pre-printed image to form on-pad and off-pad masks, it 

foregoing, that it is not necessary to acquire pre-printing should be appreciated that other means of masking or mask 

image information before every printing operation and that generation can be implemented in order to distinguish 

a database of information can be stored and accessed to between the areas of interest and other areas that may be 

obtain pre-printing image information. The database of inspected. 

information can be from a captured image, downloaded Although in the method and apparatus described herein- 

from a Computer Aided Design tool or otherwise acquired. before on-pad and off-pad images arc described as logically 

Likewise, for purposes of efi&ciency and expediency it is not ANDed with respective masks, and two threshold difference 

necessary to acquire a post-printing image after every print- images are described as logically ORcd \o effect a single 

ing operation. Post-printing information can be acquired binary image, it will be appreciated that other means of 

periodically, as necessary to keep the process within some combining the respective images can be effected to arrive at 

appropriate user defined control limits. images or an image representative of the logically combined 

While the system described herein acquires and stores an ^ images, 
entire pre-image it will be appreciated that instead of storing The previous embodiments registered a stored pre- 

an entire pre-image portions of an image constituting useful application image to a post-application image for each 

information can be stored and used to synthesize a pre- inspection. For each inspection site, a pre-application image 

image during runtime. Similarly, limited useful information is stored in memory. The memory space required to store the 
from the captured pre-image, such as a gray level histogram, ^ pre-application images limits the number of sites a user can 

can be stored and used to determine gray levels in the inspect, where each site can contain more than one region of 

post- image for binarizing. interest, e.g. solder pads. 

Additionally, while two aligned images, i.e. the pre-image FIG. 6 illustrates an alternate PCB inspection process 

and the post-image, are described herein as being effectively flow that decreases the memory requirements by calculating 

combined by optionally subtracting one from another yield- and sUaring characteristics of the pre-application image as a 

ing a difference image with pixel values that are signed one-dimensional image (i.e. histogram) instead of storing 

quantities (i.e. positive or negative), it should be appreciated the two-dimensional pre-application image, where like 

that analogs of the pre and post images, such as filtered or numerals designate like steps. The first step is to acquire/ 
re-mapped images can be processed and combined to yield ^5 construct the pre-appUcation image of the PCB prior to 

a "difference image" that has unsigned pixel values. application of the solder paste 60. 

Furthermore, while other operations are described herein Thereafter, a pre-application histogram of the pre- 
utilizing the pre-image and the post-image, it should be application image is generated 96. A histogram is a one- 
appreciated that any of the images described herein can be dimensional array of values representing the distribution of 
subject to further processing, such as by filtering using a gray values of the pixels in the image. Each of the pixels is 
gaussian filter, median filter, smoothing filter, morphological mapped into a bin, graphed on the x-axis of the histogram, 
filter or the like known in the art, in order to improve image having the same index as the gray value of the pixel. The 
quality. value of each bin indicates the number of pixels in an image 

Although a training process is described herein using a that have a gray value substantiaUy identical to the index of 
specified, known in the art vision tool (i.e. Cognex Blob 45 ^^^^ 

Analysis Tool) for obtaining characteristics relating to the Using the pre-application histogram, information about 

apertures for the stencil side inspection and the solder pads the pad regions is determined. Analysis of the pre- 

for the PCB side inspection, it should be appreciated that a application histogram is effected using vision tools with 

data base of information relating to the characteristics of the peak detection abilities as known in the art. For instance, 
stencil and the board can be obtained using other vision 50 assuming that the brightest peak corresponds to the pad 

tools, such as Cognex Caliper Tool or from a CAD tool or region, a peak corresponding to the pad region is identified 

other machinery that requires or acquires and has informa- and then analyzed with other known techniques, such as 

tion relating to the characteristics of interest, such as auto- integration, to estimate an area of the pad region or other 

mated tooling used to cut the stencil or other tooling used to characteristic information. 

fabricate the PCB. Alternatively, training information can be ss However, a preferred embodiment enhances the utility 

manually obtained by an operator and entered into the and rehability of the pre-application histogram prior to 

system. analyzing the pre-application histogram by using an optional 

It will be appreciated that although the invention model. The enhanced reliability of the pre-application his- 

described in an illustrative embodiment herein effects togram also enhances the reliability of the characteristics 
inspection of solder paste applied to printed circuit boards, 60 extracted from the pre-application histogram, 
it can be similarly implemented in processes to effect The method enhances the pre-application histogram by 

inspection of other materials such as screen printed using a model of the pre-application image to define the site 

adhesives, conductive adhesives or the like, applied on of the pre-application image before calculating the pre- 

substrates other than printed circuit boards. Furthermore, application histogram. The model of the pre-application 
although the invention is described in an "in-line" imple- 65 image, defined during training as hereinbefore described, is 

mentation with a screen printing process, the invention can registered to the pre-application image 94. The registration 

be implemented and/or the inventive concepts herein modi- aligns selected sites where the pads are on the PCB. Reg- 
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istration is accomplished as previously described. 
Thereafter, the sites are windowed and a histogram of each 
pad of the site (i.e. each pad of the pad region 80) is 
generated. 

The histograms for each pad are then summed to create a 
single pre -application histogram for each inspection site. 
The histograms are summed by adding the number of pixels 
with a 0 index from each pad histogram and assigning the 
sum as the value of the bin in the pre -application histogram 
with a 0 index. All pixels in the histogram with index of 1 
are summed and stored in the pre-application histogram with 
index of 1. The summing process is continued until all the 
intensity values for each pad are appropriately represented in 
the pre-application histogram. The resulting pre-application 
histogram reflects pixel gray values for all the pads of the 
inspection site without pixel gray values corresponding to 
the PCB region (off-pad regions 82). The entire PCB, each 
pad or each site can be processed or grouped in this manner. 

In a preferred embodiment, the area of the pad region is 
calculated and used to aid identification of the pad regions 
in the post- application image. Accordingly, in addition to 
calculating the pre-application histogram 96, the area of the 
pad region is calculated as the number of pixels in each 
pre-application histogram times the area of each pixel 98. 
The number of pixels includes all pixels represented in the 
pre-application histogram regardless of intensity value. It 
will be appreciated that the area of the pads can be found 
using other methods known in the art, such as a connectivity 
analysis of the sites of the pre-application image. 

The pre-application histogram and pad region areas, if 
calculated, are stored for each inspection site before the 
paste is applied on the PCB. Optionally, if more accuracy is 
desired, the calculated pre-application histogram data 
updates the model and the updated model is also stored. 

After solder paste is printed on the PCB, the camera 
acquires an image of the PCB (the post-application image) 
100 at all relevant sites as previously described with refer- 
ence to the registration embodiments of FIGS. 1-5. 

The previously described embodiments relied upon 
proper registration of the post-application image to the 
pre-application image to minimize errors within the differ- 
ence image. The necessity of proper registration is mini- 
mized in this embodiment. Preferably, the location of the 
post-apphcation image corresponds to the location of the 
pre-application image. However, the pad regions do not have 
to be located precisely in the post-application image because 
the pad regions are not optionally windowed as they were in 
the pre-application image or registered to the pre-application 
image as required in the previous embodiments. 

However, the entire pad region should be within the 
post-apphcation image so the entire pad region can be 
inspected and/or the calculated pad area can be used to assist 
the identification of the pad regions. To that end, the size of 
the window for each site of the post-application image is 
increased to account for errors in repeatability of the camera 
positioning mechanism, where the means to increase the 
window size is known in the art. 

A post-paste application histogram is constructed of the 
post-apphcation image within the window 102. 

The post-paste appHcation histogram of a PCB typically 
contains an unknown number of peaks because the off pad 
region's color varies and reflects hght in different manners 
as previously described. Further, if paste is present, the 
post-paste application histogram also typically contains an 
poorly defined boundary between the paste region and the 
pad region. 
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The invention determines the paste/pad boundary, if 
present, by augmenting the post-paste application histogram 
of the post-apphcation image (i.e. second-state image) using 
the pre-application histogram of the same inspeaion site. In 
S a preferred embodiment, the invention enhances the repre- 
sentation of the pad region relative to the paste region 
thereby making the paste/pad boundary more pronounced 
104. 

It should be apparent that another region, that is also 
present in the pre-application image, can be enhanced within 
the post-paste application histogram, and that the choice of 
the region, such as the pad region, depends upon the 
requirements of the application. The processing of and the 
optional use of the model would also have to be tailored to 
each apphcation. 

One manner of enhancing the representation of the pad 
region is to remove the pixels in the post-paste application 
histogram that are not within the area of the pad region in the 
first-state image, as is more fully described with reference to 
FIGS. 7A-7C. The intensity values are removed by reas- 
signing the intensity values of the non-relevant pixels to zero 
or "do-not-care" labels. To determine which intensity values 
are reassigned within the post-paste application histogram 
120, the number of pixels are summed starting from the 
highest intensity value to the lowest intensity value (i.e. 
from right to left in FIG. 7B). 

It is assumed that the pads and the paste are bright and 
therefore correspond to high intensity values. With reference 
to FIGS. 5A-5C, typically, the pad region 80 is the brightest 
3Q and corresponds to the right most peak 124 of the post-paste 
application histogram 120. Also, typically the paste regions 
84 and 86 arc brighter than the off-pad regions 82 but not as 
bright as the pad regions 80. Therefore, the paste regions 84 
and 86 typically correspond to the second right-most peak 
35 126 

Accordingly, when the number of pixels within the post- 
paste application histogram 120 correspond to the area of the 
pads (as calculated at step 96 during the pre-application 
calculations), the paste/PCB intensity value boundary is 

40 defined, such as T for example. Any pixels having an 
intensity value below the boundary, T, are set to zero and the 
resulting augmented histogram 128 is shown in FIG. 7C. 
Using the assumption that the highest gray values corre- 
spond to the pads or the paste, the intensity values remaining 

45 within the augmented histogram 128 correspond substan- 
tially only to pad and/or paste pixels. If the paste is absent, 
the boundary will be the pad/PCB region boundary and only 
contain the right most peak 124. This is one way in which 
the 0% coverage of paste is detected by the invention. 

50 By keeping only the pixels substantially equal to the pad 
area given by the pre-application histogram, the number 
peaks within the post-paste apphcation histogram is reduced 
from n substantial peaks to two substantial peaks, if paste is 
present, or one, if paste is not present. Reducing the number 

55 of peaks to a known quantity is a necessary prerequisite 
before applying most known thresholding techniques. Typi- 
cally thresholding techniques reply upon knowing the num- 
ber of peaks present in each signal prior to determining the 
threshold. This is particularly xiseful for PCB inspection 

60 because histograms of PCBs often contain an unknown 
number of peaks. For example, a PCB having a pattern, such 
as a manufacturers logo, and a PCB of different colors, can 
be processed regardless of the varying intensities of the 
off-pad region using the area of the pad region firom the 

65 pre-application histogram. 

In a preferred apphcation, not only are the pixels outside 
of the pad area removed from the post-paste application 
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histogram, but also the peak of the pad region is enhanced such as lettering identifying the manufacturer, can be iden- 

by selectively combining the intensity values of the pre- tified from the background, paste and pad regions, 

application histogram to the post-paste application histo- Further, those skilled in the art should appreciate that a 

gram or the augmented histogram, FIG. 7A is an instance of bimodal signal can be generated from any object having two 
a pre-application histogram 122 generated from summing 5 or more states (pre-appHcation characteristics and post-paste 

histograms of wmdowed pads. The prc-application histo- application characteristics) using the method of this embodi- 

gram 122 is selectively combined with the augmented mcnt. 

histogram 128 of the post-paste application histogram 120 n ^ • . t-t^ . il i_ u i - 

. ' w c u *u L- * *i_ * L .L Returmng to FIG. 6, to further the evaluation of other 

by comparme bins of both histograms that have the same . /. .f j • . . . ^ m * 

.J T c J uj- . aspects 01 the second-state image, a bmary image is created 

mdex. In a preferred embodiment, a new augmented histo- lO ^no • .i. l u j . L j • Tri^ ^ r 

1-in- * -J u uu- 4 • »u usmg the threshold identified m step 106. A soft 

gram 130 is created wherem each bm contams the maxunum . u • j • c j . j- . . . 

* 1 f r . • » threshold is used in the preferred embodiment to map the 
value of either the prc-application histogram 122 or the . „ r.^Lu- • 

* J I.- * f -J L • binary image. The soft threshold is achieved by creatmg a 

augmented histogram 128 for the same mdex as shown in . / l *l . . . . j . . 

FIG. 7D. nie augmenled histogram 120 has a subslantiaUy "^.'P '^"^ ^^"V^^ 

bimodal dLslribudon, which afstated earlier is easier for 15 ^^T. u ^'T!If f ' 

^\»r.^tu^c i« The thresholds are chosen such that n percent of the pixel 

common algontnms to tnresnold. - . . , . i . ^ ,f . f , , 

* area IS between the low threshold and the center threshold, 

The intensity value of the pad region increases within the „ p,,^,^^^ ^ ^etweeD the high threshold and 

augmented histogram to mdicatc presence of paste on the ^^^^j. threshold. All pixels above the high threshold are 

pad regions of the post-application image. However, if the j^apped to 255 and aU pixels below the low threshold are set 
mtensity values of the pad region do not change after the ^ to 0. All values in between are divided into equal increments 

combination procedure, the second-state unage has either a su^h that the mapping values will transiUon from the low to 

pad region that is brighter or has the same brightness as the ^igh value in a straight line. This is a soft threshold 

first-state image. If the brightness is the same, then 0% ^^ere pixel values that are bordering the threshold are 

coverage of paste is mdicatcd. It shoiJd be apparent that partially counted. The post-application image is then 
comparing the change or lack thereof of the augmented ^ mapped using this pixel map. 

histogram compared to the second-state histogram provides , , ^ , . ^ ^ 

information about the second-state image without further . preferred embodunent, to facilitate the ideotifica- 

processing regions in the post-application image other than the 

. ' . f ... 1. . pad region remaining in the post-application image, the 

An alternate method of combimng the pre-application ^ ^ ^ ^^^^^^^ ^ ^i^^^y ^^^^^^ 

histogram and the post-paste application histogram is by ^^^^j inspection site, similar to step 94 With the 

sumnung both histograms regardless of the maxmium val- ^j^, ^ pad region or any 

«es. The intensity value withm the resnltmg augmented ^^^^j^ j ^ y^^^^^ ^^ ^^^^ ^ ^ 

second-state histogram increases rekUve to at least one ^ q^^^ „^ ^^^^^^ ^^^^^ ^ 

other region because the mtensity values is the sum of the • • j « j ♦ j * • *u * r j 

• . r o . • i- . , . 35 region window are counted to determme the amount of pad 
intensities of the first region from both the pre-apphcation ^^^^ remaining after application of the paste. The amount of 
histogram and the past-paste appLcaUon histogram. p^^ ^^^^^^^ p^^^/f^ ^^^^ calculated by comparing the 

Those skiUed m the art should appreciate that the tech- area of pad in the binary image to the pad areas calculated 

nique of selectively combining the pre-application histo- from the prc-application image. Both the 0% and 100% 

gram with another histogram can be used alone or combined paste coverage instances are properly identified by the 

with the removal of intensity values of histogram described method. 

with reference to FIG. 7C. For instance, combining the • , ^ , . * , - ^ , 

histograms without eliminating the pixels associated with . A^'ern^Uyely, Cogoex s Blob Analysis Tool can be used 

the off-pad regions can enhance the relevant peak. Similarly, !? 'de^l^fy "^g'""- ^h^rc Cognex s Blob y^alysis 

• • f *u ■ * •* 1 c _* f *u -1 lool provides miormation such as area, center of mass, 

reassignmg of the intensity values of part of the pixels can . , , ' ^ > 

1 . J I • • «• -^u 1 1 onentation, moments, and extent, 

also be used alone or m conjunction with selectively com- ' * 

bining the histograms to enhance the relevant region. Other information can be identified within a second-slate 

After the augmented histogram is calculated, the pad image by manipulating the model, the first-state histogram, 

region in the post-application image is identified within the ^'^^ second-state histogram usmg the method of the 
augmented histogram 106. A preferred method for identify- 50 ^^^"^on. 

ing the region is to define a binarizing threshold of the Although a preferred embodiment principally described 
pad/paste region using the within-group variance method. that data representing the first-state image was extracted as 
The within-group variance method is a technique well a first-state histogram and that the second-state histogram 
known in the art and is further described in Ostu, A Thresh- was augmented by the first-state histogram, the invention is 
old Selection Method from Gray-Level Histogram, IEEE 55 not limited to such a representation of data. Data represent- 
Trans. Systems Man. Cybernet, (1987) SMC-8, 62-68, ing the first -state image can be extracted and applied in other 
incorporated herein by reference. However, those skilled in ways, such as a series of data points, for example. The 
the art should appreciate that any statistical thresholding hereinbefore-described calculation and use of area is one 
technique using histograms can be used. example. It will be appreciated that other means of extract- 
Although the preferred embodiment described separating 60 "^S without storing the first image, can be imple- 
paste regions from pad regions, one skilled in the art should mented. Likewise, it will be appreciated that the second- 
appreciate that one signal can be processed multiple times to histogram can be represented dififerenUy. 
selectively eUminate different portions of the post-paste Although the invention has been shown and described 
application histogram and/or selectively enhance an impro- with respect to exemplary embodiments thereof, various 
nounced intensity value of the post-paste application histo- 65 other changes, omissions and additions in the form and 
gram using the pre-applicalioo histogram and optional train- detail thereof may be made therein without departing from 
ing information. For instance, a pattern on the substrate, the spirit and scope of the invention. 
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What is claimed is: 

1. A method of identifying a first region within a second- 
state image, the method comprising: 

providing a first-state image having a first region; 

calculating at least one first-state characteristic represen- ^ 
tative of at least the first region of the first-state image; 

acquiring the second-state image having the first region 
and the at least one other region, the second-state image 
differing from the first-state image; 

calculating a second-state histogram representative of the 
first region and the at least one other region within the 
second-state image; 

augmenting the second -state histogram using the at least 
one first-state characteristic by enhancing the represen- 
tation of the first region relative to at least one other 
region to provide an augmented second-state histo- 
gram; and 

identifying the first region within the second-state image 
using the augmented second-state histogram. 20 

2. The method of claim 1 wherein the at least one 
first-state characteristic is a first-state histogram. 

3. The method of claim 2, wherein augmenting the 
second-state histogram includes combining the first-state 
histogram with the second-state histogram. 25 

4. The method of claim 2, wherein augmenting the 
second-state histogram includes 

combining selectively the first-state histogram with the 
second -state histogram. 

5. The method of claim 2, wherein augmenting the 30 
second-state histogram includes; 

calculating an area of the first region in the first-state 
image represented by the first-state histogram; 

removing intensity values in the second-state histogram 
outside of the area of the first region; and 

combining the first-state histogram with the second-state 
histogram. 

6. The method of claim 1, wherein augmenting the 
second -state histogram using includes: 

calculating an area of the first region in the first-state 
image represented by the at least one first-state char- 
acteristic; and 

removing intensity values in the second-state histogram 
outside of the area of the first region thereby enhancing 
the representation of the first region. 

7. The method of claim 1, wherein identifying the first 
region within the second-state image using the augmented 
second-state histogram includes 

segmenting the first region with respect to at least one 
other region within the second-state image using the 
augmented second-state histogram so as to provide a 
segmented first region. 

8. The method of claim 7, wherein segmenting the first 
region includes: 

calculating a threshold between peaks of the augmented 
second-state histogram that segments the first region 
with respect to the at least one other region within the 
second-state image; and 

mapping the second-state image using the threshold to 
produce a binary second-state image. 

9. The method of claim 8, for measuring characteristics of 
one of the at least one other regions in the second-state 
image, the method further comprising: 

providing a model of at least the first-state image; 65 
aligning the model with the binary second-state image so 
as to produce an aligned second-state binary image; and 



measuring characteristics of one of the at least one other 
regions using the afigned second-state binary image. 

10. The method of claim 7, fiirther comprising: 
providing a model of part of the first -state image; and 
measuring characteristics of the segmented first region 

within the second-state image using the model of the 
first-state image. 

11. The method of claim 7, further comprising 
measuring characteristics of the segmented first region 

within the second-state image using the at least one 
first-stale characteristic. 

12. The method of claim 1, fiirther comprising: 
providing a model of the first-state image; and 
performing a registration of the first-state image to the 

model to form an aligned first region model containing 
substantially only the first region; and 
wherein the calculating the at least one first-state charac- 
teristic representative of the first region of the first-state 
image includes: 

calculating the at least one first-state characteristic 
substantially of only the first region of the first-state 
image within the aligned first region model. 

13. The method of claim 1, wherein the first region is a 
pad region, and one of the at least one other regions is an 
apphed paste region. 

14. The method of claim 1, wherein the first-state image 
is part of a printed circuit board before application of paste, 
and the second-stale image is the part of the printed circuit 
board after application of paste. 

15. The method of claim 1, wherein the first-state image 
and a second-state image represent different manufacturing 
stages. 

16. The method of claim 1, wherein the first-state image 
contains the at least one other region. 

17. A method of identifying a first region within a 
second-state image using at least one first-state characteristic 
of a first-state image, the method comprising: 

providing a model representative of characteristics and 

position of the first region; 
identifying a first region within the first-state image using 

the model; 

calculating the at least one first -state characteristic of the 

first region once identified; 
acquiring the second-state image, the second-state image 

differing from the first-state image and having a 

unknown number of other regions; 
calculating a second-state histogram representative of the 

first region and the unknown number of other regions 

within the second-state image; 
augmenting the second-state histogram using the at least 

one first-state characteristic to provide an augmented 

second-state histogram that substantially eliminates 

values in the second-state histogram representative of 

substantially all but the first region and a second region; 

and 

calculating a threshold of the augmented second-state 
histogram so as to substantially identify the first region 
relative to the second region. 

18. The method of claim 17, wherein the at least one 
first-state characteristic is a first-state histogram. 

19. The method of claim 18, further comprising 
combining selectively the first-state histogram with the 

augmented second-state histogram before calculating a 
threshold of the augmented second-state histogram 
which substantially identifies the first region relative to 
the second region. 
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20. The method of ckim 18, further comprising 
combining the first-state histogram with the augmented 

second-state histogram before calculating a threshold 
of the augmented second-state histogram which sub- 
stantially identifies the first region relative to the sec- 
ond region. 

21. The method of claim 17, further comprising 
binarizing the second-state image using the threshold to 

produce a binary image having the first region seg- 
mented from a remainder of the second-state image. 

22. The method of claim 21, further comprising: 
aligning the model to the binary image to produce an 

aligned binary image; and 
calculating a change in the first region between the model 
and the second-state image using the aligned binary 
image. 

23. The method of claim 21, for calculating a change in 
the first region between the first-state image and the second- 
state image further comprising: 

updating the model with the at least one first-state char- 
acteristic once calculated so as to produce an updated 
model; 

aligning the updated model to the binary image to produce 

an aligned binary image; and 
comparing the binary image to update the model so as to 

calculate a change in the first region between the 

first-state image and the second-state image using the 

aligned binary image. 

24. The method of claim 17, wherein the first region is pad 
and the second region is paste. 

25. A method of using a vision system to identify pad 
regions of a substrate within a post-paste application image, 
the method comprising: 



providing pre-application image data representative of 
characteristics of the pad regions before application of 
paste; 

calculating a pre-paste application histogram of the pad 
5 regions before the paste is applied on the substrate 
using the pre-appUcation image data; 
acquiring a post-paste application image of the substrate 

after the paste is appHed on the substrate; 
calculating a post -paste application histogram of the post- 
10 paste application image representative of the pad 
regions, substrate regions and applied-paste regions; 
augmenting the post-paste application histogram using 
the pre-paste application histogram by changing the 
representation of the pad region relative to at least one 
J 5 other region to provide an augmented post-paste appli- 
cation histogram; and 
identifying the pad regions within the post-paste applica- 
tion image using the augmented post-paste application 
histogram. 

26. The method of claim 25, wherein the at least one other 
region is the substrate region, 

27. The method of claim 25, wherein the at least one other 
region is the paste region. 

28. The method of claim 25, wherein augmenting the 
post-paste application histogram includes: 

calculating an area of the pad region before application of 
the paste represented by the pre-paste application his- 
togram; 

removing intensity values in the post -paste application 

histogram outside of the area of the pad region; and 
combining selectively the pre-paste application histogram 
with the post-paste application histogram to provide an 
augmented post-paste application histogram. 



20 
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